Nonsmoothness and the Bfgs Method∗

This paper investigates the potential behavior, both good and bad, of the well-known BFGS algorithm for smooth minimization, when applied to nonsmooth functions. We consider three very particular examples. We first present a simple nonsmooth example, illustrating how BFGS (in this case with an exact line search) typically succeeds despite nonsmoothness. We then study, computationally, the behavior of the BFGS method with an inexact line search on the same example, and discuss the results. In further support of the heuristic effectiveness of BFGS despite nonsmoothness, we prove that, for the very simplest class of nonsmooth functions (maximums of two affine functions), the method cannot stall at a nonstationary point. On the other hand, we present a nonsmooth example where the inexact-line-search BFGS method converges to a point despite the presence of directions of linear descent. Finally, we briefly compare line-search and trust-region strategies for BFGS in the nonsmooth case.